Yeah. So this was that face to face meeting. Um but it uh it doesn't There isn't th so there are things in the eye tracker record that we definitely don't want in the G_D_F_ format, um like the frame rate eye movement. You know, we we uh uh i we're if we're expecting the G_D_F_ format to be um sort of a parsed version of events. And you know there's just too much raw frame f frame rate stuff coming out of this. You wouldn't prob you wouldn't want it in this kind of format for up-translation to the other things. Yeah, we yeah, you can get back to it. Yeah. We're not throwing anything away. This is just the stuff Yeah. Yeah. But the th but the way to think of this is this is the data that we want to be able to analyse against the other tracks of data. So the data that we wanna compare with a language or with whatever. And um so Yeah. S so in essence what you would then do is Well we're not doing frame rate at any frame for well, maybe we are. I don't know. I I had thought we were yeah, the eye tracker does. But But right. W what w um the kinds of events that we had r uh before talked about putting into the record for then use in ELAN and N_X_T_ isn't based on saying every X_ seconds something. You know, give me give me what's happening every X_ seconds. It's more like, you know, give me the fixations and the um the blinks. Yeah. Oh yeah. Durations are in there. But it's not there's a difference between um saying that something is in a particular state every frame, whatever the frame rate is, you know, ten seconds a minute. You know, that's that's one kind of w way of looking at data, and um a parsed version of the data, which isn't at um any particular length. It relies on this really close frame rate that's underneath. No, no. The time the time is still in there. But you'll get uh what you want is to figure out what are the um the concepts behind that data that you want represented. So rather than saying the eye was at this particular place, you know, here, and then a tenth of a second later here and then a tenth of a second later here. You say there was a fixation from this time to that time and there was blink from this time to that time. And Okay. Mm-hmm. Right. So there's two issues here. We're not throwing anything away. But the question is what tool would you use to get that information out of the data? Would you be so the more you can plan Mm-hmm. Yeah. Mm-hmm. So you need to know the percentage of time it was on during that fixation. Yeah? Okay, so the percentage is i i is a cut it's not something you're using for measurement. It's a cut-off for whether or not it counts as a fixation? Is that Mm-hmm. Mm-hmm. Mm-hmm. Right. Yeah. 'Cause all you wanna know is whether they were looking at the same object in the period after. Yeah. Okay. Can we draw it to make sure that that um that this is satisfying Ellen's concerns? Well, there's a white-board, right. We love this thing. See if the pens work. So um this is um A_'s eyes, right? And B_'s eyes. And I think what you're saying you should probably draw this right, is that they you know, if um from this time to this time they're looking at triangle one. W what you're saying is that you want to know in the critical you wanna know after they're they're looking at triangle one what's happening in this period with this guy, right? Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Yeah. So Mm-hmm. Alright. So Yeah. So, you know, he might get triangle one there and no maybe this one's gonna overlook lap quite a bit. So what you want to do is be able to define um bigger periods, which is the period when they're sort of interested in triangle one overall, right? This is sort of some meta level analysis. Yeah. But that's that's not a something that you can do in the N_X_T_ query language, but it you can't do that in any you know, this is such special purpose s but it's easy enough given the data format for any of these things to do that. So you could do it on 'cause i it's just a matter of i the hard part is deciding how close together these have to be before you decide that this is a event that you wanna pull out. Because, you know, algorithmically you're you're already putting together I mean this is essentially already, you know, these fixations with stuff in between. And uh um, you know, we we've got an algorithm for deciding when that's a look, I guess. You already have that or no? Oh yeah, yeah. So it's all fixations. They're these are all fixations and saccades, but within the yep. So that's easy enough. So the hard thing is they could be moving their eye arbitrarily here, right? And you know, maybe even having fixations on other things? Yeah? Yep. Back and forward. Yep. So um this isn't yeah. But what you need to do is we we can already build into this G_D_F_ format these regions if you can give a definition of what you think that is. But I think the right way to do this is to be able to inspect the data in um some tool and play around with the definitions. Because you won't get it right the first time. Yeah. Mm-hmm. And uh ELAN's a reasonable choice for that, right? What you need is something well no, because you need to Y you need to be able to see the effects of this. So I g I guess this is a case of Craig writing some scripts so Robin having some ideas about what the relationship is and saying, you know, add this data a automatically. It I mean it's not that hard, right. It's just there'll be a bunch of these, you'll have some rules about how maybe how long you spend on other objects and w whether the other object is um the existing construction or not, right. And so um Ah, right. So if they if they uh they had a fixation on C_ you wouldn't be worried. But if they had a fixation on Yeah, they're yep. But yeah, they're gonna look at the clock um and they're gonna look at they may look at completely irrelevant Mm-hmm. Yeah. But that's like a separate analysis, right. So It no no. The so every separate Yeah. Mm-hmm. Yeah. But what we find e what we can get out of the data easily at the moment is at this kind of level, right. And then it's a case of defining algorithmically all these other transformations that you want. So this is one uh we hadn't thought of adding before, but we should, right. Which is um well, how would you decide whether something was the current addendum uh the cu So that's a human decision. Yeah. Um But the ex But we believe But we believe J_P_ to be defining that maybe. Yeah. Mm-hmm. I could believe them, yeah, no, taking an interest in that. I mean the general division was language here and other stuff there, right. I'll just I'll just minute that. Oh yeah, eye tracker here. Yeah, I mean we know that. Well yeah. But I mean with transaction coding on something like this you would use i y you wouldn't consider it verbal analysis exactly. It'd be verbal plus action. Because it's a task breakdown. It's just a segmentation. Yeah, but you wouldn't have you you wouldn't do two codings for task level one you wouldn't do transaction coding in a action segmentation. I I don't think that's realistic, because I think what you'd actually do is use um the full record of what you have and do a segmentation. I think it'd be hard to understand what was going on if you used like just the language without watching the video to decide whether we're breaking down the task. Oh, you don't want it to be coding th uh based on that though and then that seems a strange way to go about things. I mean this the language events are multi-modal, right. You know, they're they're doing all these things together. Wha why do you care whether they can do it just based on the the language? Is Well, it seems to me much less important than the other things one could get out of this data. But I, you know, I it's a possibility. We'll Yeah, well I c I can see maybe wanting to know whether you can do the chunking just based on the actions without the language in cases where they use both. It's more g I can't quite see why it's important to know that they can do it just based on the language when w you know that they had both. Uh-huh. Well, I think that one's just bound to fail. But yeah. Really? Okay. Anyway. So back back to this main problem, which is yeah, w the record that we are getting in the first instance is about the m about these you know, they're looking at some region of the screen defined dynamically. And then we need some way of knowing what the you can you can see adding these other analyses about, you know, they're they're jointly focused on this region and trying to figure out what s percentage of time they're looking at it as adding new tiers of information to either the ELAN track or N_X_T_. They're they're both sort of track based in this way. Um but the hard part is knowing how you wanna do that. And I I think in the first instance what we're trying to get into the G_D_F_ format is just this. And then we've got no option but to uh figure out a way for like to look at this, just explore this data, suggest ways about doing it and um be able to play them back and see um when we think we've got these things right. I don't think we even know what the set of these things are that we want, much less how to get 'em yet. So th Yeah, that's right. So this is not the you know, this is something we're aiming for. But this isn't something that affects what Craig programs for the initial G_D_F_ translation. Um Yeah. So let's just return to this question of frame rate. Because this is the thing we were not planning to um transfer into this format. So again, you know, what a frame rate kind kind of coding is ag it can again be seen as a track. But it says it's a coding like this, right, where you say it's in state A_, state B_, state C_, state A_. Right. So you right. So I'm just you're happy that it's not gonna have frame rate uh like this. It's gonna have an interpretation like this. What does that mean? Sorry. Right. W right. But it seems to me it seems to me that Ellen's concerns might mean we need to add more information to these tags because um, you know, this thing in itself is th bunch of fixations, right, with saccades. Fixations and the percentage of time those fixations cover, you might want. Okay. So number of fixations. Percentage of No no. This is this is still um uh summary data. They're he's we're talking about just adding attributes to these things that say the number of fixations that counted as that looking at triangle one, right. So y it's possible Yeah, the spots'll be different anyway. 'Cause i you're not gonna g have the same pixel. So um we have the option that we can put in an A_ fixation thing here, right, as well. If you want the smaller if you want the smaller coding in in here. So that it's not just parsed into this idea of which object. But you also have the raw fixation um data. We can put that in as another track. If you think that that's something that you might want to look at in one of these tools that shows you the tracks against each other. Yeah, I d that's all he asked for. Yeah. Well, I think Yeah, and in N_X_T_ there's no cost, right, because the y the uh you just don't choose to load those. All we're doing is dumping it as output that you can load if you choose to. Um i g s you know, so if they're things that you just know are crazy and you're not gonna want then we don't do it. But otherwise we go ahead and dump 'em. 'Cause it's easy to dump out the fixations. Um yeah? Okay. So it's the number of fixations the did you want the you M not e well, if it's each, you have to d you have to break it down into another tier, right. You can't you can say there were three fixations and they averaged uh a certain time. But you don't wanna D well, what about the the overall sum of the durations? Is that I mean which way do you want to Yeah, then that's that's the way you want it? That way around? Okay. And uh y did you want the percentage of time fixated as opposed to which is sort of another view of this number. You can have as many of these as Yeah. Well, it's not um the the extra cost of having these as attributes is not high. So um, you know, it saves you i uh having to do write scripts to do arithmetic on that later if you know that these are numbers that are gonna be useful to you. Mm-hmm. Yeah. I don't know. Well well but y those are different tracks of information. So what you want is one track of information about which object it's one, one about whether or not it's on the other guy's gaze, and one about whether it's on um one of the mouse positions. Yeah, but i in in the analysis you should treat those as completely separate tracks of information. Because they can be looking at they could look sim simultaneously at somebody's mouse, somebody's gaze and some object, right. So Yeah. So the these are independent. So so what we want is whether they're looking at s at the other person's gaze, right, and say they are in this period well, I mean uh they're probably not given what I've trying to Mm-hmm. Mm-hmm. Mm-hmm. Yeah. So g so there's there's a lot of tracks here, right. Because well we just added this fixation track. And then there is A_'s mouse, not their eyes, is on an object, right. And the mouse might be on triangle one. But it doesn't matter. I mean i the mouse we're just saying where is the mouse. You know, it's on this object. And then um A_'s eye is on well A_'s mouse I guess, right. A_'s eye might be looking at A_'s mouse here. A_'s eye might be looking at B_'s mouse here, right. I mean these are all the different ways of taking cross-products at the things that could be co-located. So you just treat 'em as independent. And I'm guessing from what you say yeah. I guessing from what you're saying you want all these tracks. Yeah, which is fine. There's no problem with that at all. Yeah. S so um uh you kinda got the idea here? Yeah. But the important thing is you don't treat that entire set as mutually exclusive and exhaustive. It's just so tha the objects is like a separate level of analysis. If whether they're looking at the objects than whether they're looking at the gaze. 'Cause they can happen at the same time. Does that happen? Oh, right. Uh-huh. Okay. So what you're saying is like a new problem for us, which is I'm gonna change That's the bad one. Yeah. So you're saying Okay. Can you write on my pen pad? Just say, yeah, pens. White-board pens. So you're saying, you know, they could be looking at um uh square one at this point. 'Cause they close together. Okay, that's a problem for the data models and either of the things that we were using. What do you mean by greatest overlap? Draw uh my g my uh spacial reasoning is no good. See uh I I'm the one that uses the white-board. 'Cause I actually put the the microphones on. Uh this is the better pen. Uh-huh. Yeah. But you had a way of choosing which one, right? So you just mean which i is it closer to the is the circle I mean, what is this circle? They're looking at a pixel or something, right? Right. S right. Oh, so those vary in size. Yeah yeah. Mm-hmm. Yeah. They could be. Oh, but what pain that's gonna be analytically. Because it's gonna how many parts are there? Like Okay. I know ways of getting around this analytically. 'Cause you c If they're really close to each other, it's gonna get foxed. Yeah. It's not the data format I'm worried about. It's the um the way you do the analysis. Because you don't wanna have to say um, you know, did they jointly look at triangle one. Okay. Did they jointly look at square one. Okay. Did they jointly look at you know, you need some way of uh uh going over the whole thing. But i c we can find ways around that. I think for now we we do it this way and then we we think about what we need out of it in the end. It's just you know, in ELAN it that has the side effect that if you go for the naturalistic way of of up-translating to ELAN, there'll be a zillion tracks. And it'll probably ruin their viewer. 'Cause you'll get very sparse data on each track. It'll be like the old referring expression generalis uh uh visualization um the map task. You know, where they talk about one object one landmark and then another landmark. Well, it's big, right. Well we'll see if ELAN likes it or not. Yeah, and we b we better also like have in the representational list of par like even in the N_X_T_ up-translation a list of parts. It it affects the way we do the N_X_T_ u up-translation and make things easier for people. Well the And it suddenly becomes and there could be more than one construct at a time, right. It's there is no the construct, right. What why does it why does construct one become construct two when you add something to it? Yeah. Oh, and this is because we don't want it to stay triangle one when you start with a construct it's not a construct at all, it's just a one thing. And we don't want triangle one to suddenly have bigger stuff. Yeah yeah, there are. There are. Mm-hmm. Yeah. That was the next question I had. Yeah. Oh yeah. Mm-hmm. That'll cover most of 'em. Mm-hmm. I yeah. It's triangle two. You don't because that's you'd have to have human coding to do that. Yeah, there might be multiple triangles that you've broke and you won't know which one they're planning on using it for. Well you know which mold they come from. And so you know which shape they match. But you can't possibly know which one of the ones they're meant to replace. Oh, so there's no two parts with the same shape? Then Oh same shape and colour. Right okay. Oh okay. So you can tell what it's meant to replace then. Because it's the same it's uh just a case of the mold, right? yep. Yeah. But the this also Mm-hmm. Mm-hmm. Yeah. We in when you specify these things, do you specify the molds or does it figure out what the molds are based on? So is this hard-wired into Tim's program? Or is it just an artifact of the way we s Right. Okay. So it's hard-wired. So minor modification needed for Mm-hmm. Uh but what were the req wha Has multiples, yeah. Mm-hmm. Mm. Yeah, uh but they don't have enough to make unique Mm-hmm. So what Yeah. Mm-hmm. So I th I think we've gotten to the point where, you know, to try to summarise, uh we we think Tim's program can do both of these conditions that you want. But I'm still worried about what the uh effects are for analysis. Because you were aiming at something in um the G_D_F_ format and I wasn't quite sure what. And w Yeah. So so okay. So I'll summarise what we said about that part that um every time you cast off a new part from a mold or, you know, every black triangle has a different I_D_. And you know it's a black triangle 'cause you know which mold it came from. But you don't know which part it was intended to replace on the screen, because you can't mind-read. What do you mean define? Mm-hmm. Mm. I think that this is a human coding that is part of the action coding. Because I don't see how anybody but a person watching this can guess why they cast off this thing at this time. Yeah, but that's okay 'cause they all have different underlying part numbers. And when you say what a construct No. No, no. Yes. Mm-hmm. Yeah. So we can easily look at things like that automatically. It's just the why they g why they get new parts that's the problem for us. We can't if they suddenly if they decide to get a part of a mold we don't know we can't know why until they do something with it, if they do. Oh yeah, you said that. Um Yeah, but you can But you can break you you can but you can break one when you break one, you don't cast off the mold yourself, it happens automatically? Oh okay, never mind then. So they just have to use it, yeah. Uh-huh. Oh, but they have to drag it up past the mold line. Yeah. Mm-hmm. Yeah, where there's extra parts. Yeah. Okay. So this why question doesn't even arise because 'Cause you know what they're gonna do with it. When they decide to move it past the mold line, they're p doing something with it probably and you'll know what they're gonna do with it because They do it. Yeah. How would you do that? Yeah, but there's there'd be no way of coding that automatically, right. How could you possibly do that? But that's a human coding, right. Mm-hmm. So w So uh that's gonna particularly be a problem for J_P_, right. Remember m all the things we're looking at are f with language with and without language. And it's gonna be very difficult to tell whether they broke something intentionally without the language. Which means it's probably not Yeah, okay. But th um this is properly part of the action coding. So we should well we should make a wish list, right, as a side effect. Mm. they could discuss deciding to break something and and then one guy does it. Yeah, yeah. Mm-hmm. No, no. It was them just dr uh forgetting to um define an attribute they had to Well, whi which is probably a documentation fault, right. Yeah. Yeah. Is there a y yeah. Well, while we're on past things, is there anything else you wanna tell us about the prior part of this, which is the, you know, what else is he holding you up from Uh-huh. Right. So it doesn't slow down the eye tracker. It doesn't do anything nasty. So we we p so we pay for it and Nothing important and noth nothing big enough they're gonna look at it, right? Yep. Yeah. Yeah, yeah. I know. Oh, do you need two copies of Camtasia to do that? So you're gonna look into this, right? Well y c look at the licence conditions, it might not I c I never pre-judge licences until I've read 'em. So uh well and um hold on, before we get go any further. So this has implications for data storage, right? Because w um it means we've got well no, this is three times, right, 'cause we've got oh no, we're gonna build we're gonna use JavaScript to build the videos on the fly, right, 'cause it's fast. So the the permanent storage is just the Camtasia stuff. So uh is that a problem? Where are we putting things? Oh we can we can put 'em in the same thing, right? You can dump 'em into one video frame if you don't mind losing resolution. But that's maybe a problem, right, because So the synchronization's gonna be a problem. Yeah, before the task. No. So These are ba these are back-ups. Right. So the flash get allows you to hand-synchronize them later if you need to by stripping extra video off the front. As long as you make sure you start 'em before the flash then you're fine. But it will there's a a cost of having to go to back-up, which is synchronizing them. Oh okay. So you so you should start the Camtasia really early then. Right. Okay. Uh-huh. That's not where you're putting it. I mean u Well okay, you've got the proper sound record. But you're not dumping it through Camtasia to record it, right. You're What before we were gonna use Camtasia at all, what well what we were gonna do what were we going to do with the sound? Uh-huh. So uh ha wou is there any sound degradation that comes about from putting this through Camtasia rather than running it out straight? Okay. So what you're planning on doing it is bunging it on one of the Camtasia tracks and then splicing it off the Camtasia track. It's an easy bit of yeah. And then you're gonna bung it on the other Camtasia track too if you need it, right? But we'll just we're gonna store all three separately then. And the sound is going to start at the same time as one of that arbitrary one of the Camtasia videos. But you'll know when the real experiment you'll know the relationship between that and the eye tracker timings? Yeah. No human intervention required. It's time-stamped. Yeah. But ho hold on. Do we need that? I mean it I I thought the way that Tim had this set up originally, um the audio uh w the time stamps used there would be joint time stamps between the audio and the eye tracker, so that, you know, uh the audio record ten seconds in was the same as ten seconds into the eye tracker record so that we didn't have to do any extra hand work, you know, any uh chopping the starts of audio signals to get there was nothing okay. So we're relying on your MATLAB script to get us the chopped version of the audio, right? Right, so we can either adjust the eye trai track data or we can actually chop the audio signal to take off the first whatever, however many seconds. Yeah, but uh but d um the uh lining up the Camtasia two is gonna take we w he's working on a MATLAB script for the sound but not for so the one that's got the sound on it, that'll also tell you where to chop the video to get it to line up. The one that doesn't have the sound on it, it won't. Oh. So it's just got the bleeps. Oh okay, so w no hand synchronization required at all. Um as long as the scripts work, right. 'Kay. How confident are you about the finding these things? So we just have to w hope we don't have an any subjects with the odd vocalizations. Yeah. Okay. Yeah. 'Cause this this is the drop-out test. Yep. Okay. Yeah, yeah. Mm. Oh yeah. I know but so you g you got a solution for that then? Oh right, okay. Yeah. You're sure there's not a better solution that involves 'Cause, you know, people might wanna use this for speech recognition or something. You never know if it's the data is there then You know. Yeah. And you're gonna want forced alignment. Oh no. Well no no. Better to collect and sort. Uh well it's the mono Wait wait, w I I thought the sol that a better solution was fixing this problem with the mono microphone socket. Sound card? So you said the problem was the m it the m m i is expecting a mono microphone input. So is that fixable? Like is there any way yeah. Uh I think a new sound card sounds a better solution than monkeying about with trying to filter white noise out. Which isn't hard. But hi but it's only got one input, right. I believe it's easy to put two monos into a stereo signal. Yeah. So Yeah, yeah. So is there any reason why we wou don't do that? Yeah. Well, it's in the old days we probably would have want a little if they'd been in the same room we would have had a little stereo thing in the background as backup as well just to make sure. 'Cause it's slightly dicier you know, if his scripts don't work then you don't get the overlap. It doesn't have to be a real trial. It just has to be two guys talking in the two bits. And then We've both been so badly bitten in the past. Yeah. But this this sounds reasonable to me. I so I think Yeah. Do do they both have the same sound card problem? I wouldn't count on it. Okay. Oh Well uh yeah. I think tell the guys. To yeah, yeah. Yeah. Mm-hmm. Yeah, it's just the way they wired it. So um it's getting late and I've got a two o'clock and I've got a nice lunch. So I wanna eat my lunch today. Uh what did you wanna get through today more? I mean I think I think it was useful going through your expectations about this. 'Cause um that's quite a bit clearer in my head at least and but I think mostly we leave you to go away and re-design the G_D_F_ or add new bits to the G_D_F_ along these lines. And then w Yeah, yeah, yeah. But I think the thing to do is um for you to go away and think about it this way with the different tracks for the different objects and the um the kinds of things we've added about whether they're looking at the other person's gaze and what have you, and and try to change the spec so it it reflects this and maybe um oh maybe by that point there will be some sample data or something uh and the right thing to do is for us to look at the sp the spec the way you understand it now and some data. And then Ellen'll have new ideas about what's needed or about these post-analyses, you know, the and then and we can add 'em in. But it's gonna emerge over time. I mean that's clear. Oh, so the spec to the other conversation, yeah. Yeah, that's that's solved now. Yeah, test. We have to test. Well you don't have your models, or do you? Yeah. Yeah. Well, uh anything you actually need fixed. You know. Joe doesn't live here anymore, right? So Well, i but he wasn't gonna be booking any time to us after the thirty first of October. So um that that means that any bugs that need fixed are now Craig's responsibility, right? You don't know have any change on that, right? 'Cause he that's what he said to us originally. So I would assume, you know, no more contact with Joe. It's not fair to get work out of somebody for free. So you know, ask Craig instead. Yeah. I don't think Well w he wasn't paid in advance. He's being paid in arrears. So it doesn't matter. Uh I think it's I think it's better to have all the software. I mean it's not big things, it's just maintenance at this program at this point and Yeah. I if he can give it, you know, like because the problem is you gotta have an immediately when you discover a bug in that. And we can make Craig shift all his other priorities to do this 'cause we bought him. Right. Yeah. So it's a kinder way. It yeah. Yeah. But it sounds like you don't think there's anything that actually needs fixed at the moment. Because w we only need this to work well enough that you can use it for something and you got work rats. So And how many models do you need to completion? And how long does that take? And you got And you know about complexity? You're waiting on Marloes. You can cold call her today, this afternoon. Mm-hmm. That's her job. Come on. Oh yeah. Okay. So you can dash in there for half an hour at a time? Doesn't sound very useful for you. Uh-huh. Okay, well. You guys sort it out. Mm-hmm. Right, but it would be kind to tell her ahead of time which of your morning sessions you're bumped. Yeah. Mm-hmm. I thought that was the problem that your mornings weren't sliced off on the booking system. 'Cause you know, i if you're sa told use the booking system Yeah. And what are the instr what are the people upstairs downstairs have um i you know, as rules about who you hand the keys to. 'Cause, you know, we like people to know to ground the oh I thought the the lending for the booking system Caroline. See, 'cause you're supposed to uh have training in Oh okay, so that's the safety. And you control those accounts. Oh okay, that's fair enough. Okay. Uh-huh, yeah. That's safer. 'Cause then they learn the first time. Yeah, yeah. Okay. I really have to get out of here. So um what else is important before you run? Nothing, right? You got uh willing heads. Oh, I don't think I brought my diary up. Uh but maybe. Why? Okay. So Yeah. Mm-hmm. Hmm. So do you know what time of day you might want? 'Cause I should c I should check that. Okay. You all have to b you'll have to be there to when it falls over. Yeah. Okay. So that all sounds f good fun. Well I can just leave you guys here, you know. Why not.
Which data. You know, I I want everything and then I'll decide what I want. Okay. We can get back to it. We can get we can always get back to it. We're not th actually throwing anything away. And if we decide to re-jig this at some finer Mecha method. We'll still have the original and a method of uh of filtering. So Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Okay. As long as the format in which we pick it up is a format which could be generalised to a finer take. Because the worst thing that can ever happen to you is you discover that you've done huge amounts of analysis and th you actually had the data but you threw it away because you treated it so grossly. Okay. So whatever we should be able to b you know, backtrack and say okay instead of every second, every tenth of a second or some such thing. But you know th Well, the eye tracker does. Mm-hmm. Well, you need Well one needs to know the duration of these things, right. So one needs to be able to say Mm-hmm. Mm-hmm. Well so there's no real time, there's only ev there's only events. What I'm we're trying to figure out is whether we've thrown time away. Here Okay, so one of the formats in which eye tracker data is analysed is percentage of time spent on some target as opposed to some competitor over the first second after some event, okay. So you actually have to be shu have to show at some time slice rate whether the eye was on the same target as the other guy, a similar target, what we'd both been deali you know. So we have to b we have all the parts, the interesting parts of the screen identified and be able to show distribution of gaze over time. Alright. Now we could be really unlucky and somebody would expect us to do that at the real frame rate. But I think that's really unlucky. But but my point is that we mustn't throw away in or lose the capacity of being able to deliver that kind of data. Mm-hmm. Mm. S so typically the eye will move, right. So over any one second, they eye's actually f fixing on a bunch of different things. And you and so when you say the eye was on this from here to here what's gonna happen is that is that if you use the real frame rate, it's gonna jiggle all over the place. It's gonna be on this landmark. Uh it's on this object, that object, this fixed thing, that fixed thing, okay. You're gonna get lots and lots of stuff. And it's percentage distribution that you're going to want. It's not uh I went here. I s you know, it's not like person walking. I went here, I stayed here. It's it's more like a fly hovering. Mm-hmm. Because if you if you take them as separate events you get thousands of separate events. Mm-hmm. Against bigger frames. Well, that's the that's the problem. There are two ways of doing it. One is that there's always a cut-off, right. But the other is that there will be a time span, a kind of reaction to some event span. Right. Right. And so I mean there are there are really two ways of looking at this. One would like to know for example the percentage of time overall in which two people are looking at the same thing, okay. And one would like to know sequences of they didn't look at the same thing and then they broke it. Or they did look at the same thing and then they did or didn't brea you know. One one could imagine those two categories. That's that's a kind of testing of our hypotheses. But there's a certain amount of um dues you pay to the way they do things in the literature to get your papers published. And one of the things they will want to know is an event series after some critical event, right. What percentage of time is given to looking where the other guy is looking. And what percentage of time over some, you know, reasonable time span of a couple of seconds, alright. So they'll be they they'll want to see essentially the gaze settling on particular incidents on particular places. So we have to be able to deliver those two things and they're rather different demands. Right. Or how l alright. Let's let's imagine a typical construction event, okay. There is some part there are there are separate movable parts on the screen ready for use, right. There may be or not a construction already begun, right. Somebody makes the first move. There's some kind of communication either gestural, imaginary or verbal, that plans how we're gonna do it, right. Before that happens the person who speaks is going to do some kind of visual scanning. While they're speaking or communicating in some way, the other person may or may not be looking where they're looking, okay. They may be overlapping gaze at particular objects which are of interest. Let's call it the c the construct, existing construct, which could be zero. Alright. Or the um the addendum, the thing which is next going to be added. Next piece, okay. Now in neither case, when you draw this, he's looking at triangle one, which you're actually drawing, you know, for a the period of time in which he's steadily looking at triangle one is gonna be very very short, right. Because staring fixedly without interruption, blink or or saccade is is an extremely short event. That's fine. So it's in a region it's in a region. Okay. But even so you're likely to get bouncing in and out of the region, right. So even so you wanna look at if we're interested in how long before the construction move takes place, um how much of the time they spend looking at the same thing. Let's call that our measure of alignment. So they're gonna be yeah, that's right, A_ is gonna be on triangle one and various other places. That's right. And and B_ um okay. And we're gonna look at the percentage of of that time where they're both in the same region. Right. So what I want to make sure is that we don't um simplify, do you know what I mean? Temporarily simplify too much, so that yeah. 'Cause then you lose percentage as an as a as a D_V_. Right. And so B_ has some periods of looking at one and all it will which will also be intermittent. Because Right. Mm-hmm. When they should be. Right. Okay. Mm-hmm. Well It Right. What Yes. Yes, because for example for part of that time both B_ and A_ are probably looking at the construct, okay. If if your intention is to move the red triangle to sit on top of the thing already constructed, you would tend to look back and forth. Right. And we should be able to to figure that out. Yeah, sure. So so the question would be under what tool should we be looking at these events. So Right. So so essentially there's there's um A_E_ is divided into T_R_ one and C_ and other stuff, okay, where C_ is the construct, the existing thing. Well no. I mean we'd take a separate we'd look at the percentage time I mean I have no idea because I don't know how people look when they're building things together, okay. So there's a there's the addendum and the and the co the existing construct. And I don't know whether they're gonna spend more time looking at one or the other. But if they're whatever it is that one is looking at, if the other's looking the same place they're in good shape. Yeah. Sure, sure. And that's what the other stuff that's what the that's what the diagonal stripes are for. That's true. But we c we can one of the things we'll be doing is um categorising people I take it or interactions by the amount of time people spend looking at the clock. We'd expect that if we put people under time pressure they'll look at the clock a lot more. Sure, it's a separate analysis. But you don't wanna throw it away, right. So so you can define if all of these things are actually categorised by the eye tracker as to where the eye is, alright, you should be able to pull out any interesting category and say alright for this phase it's T_R_ one, or C_. For this phase it's um S_Q_ one or C_, okay. Mm-hmm. Mm-hmm. You have to actually transcribe the thing or watch the film further on. Yeah. Yeah, absolutely. Yeah. Right. Right. Yeah, that's the addendum in the construct should be, you know, kind of they they change their true identity. But they're i their categories of stuff now I w my kind of assumption was that that this was J_P_ land, that that's what he was re what he was really interested in was the kind of coding which would define the building sequence, okay? So and the and maybe he wants to think about that more and say well there were several candidates for the addendum, alright. And and it was the negotiation of which candidate was gonna be the right candidate that's actually the interesting stuff. So I should talk to him about this. He's he's actually um emailed me with a l with a list of things he wants to discuss at length. So I w I'll bring this one back to him. Well this is my question for him. Is he gonna define this kind of building sequence in a way that we can get out. Or is he so b I mean, you know, they've been working on construction there for a while. So it may be that they have a coding system that's ready to go and we should just apply it. Okay. Um they do a lot of sub-assembly in the tasks they've been doing. So they may well have a yeah. Let's find out. Right. Right. So wha right. What well we but we have the now actions I mean actions might mostly be theirs, right. But since we have the the eye tracker i th we're gonna have to answer their questions about gaze, right. Yeah, I right. But um so they'll they'll uh they'll tell us something about that. But um presumably what you piously hope is happens here is what happened with um transactions, right, that the verbal analysis and the um and the visual analysis give you the same breaking points, the same chunking points. Well, I mean what you know. It's it is it is a task breakdown. But for the verbal version there are ways of announcing that now we're doing a new one, right. Or that we're finished with the old one. No, you wouldn't. But I mean one of the things one of the things you'd like to establish you know, as an as an outcome of this is that you could analyse either end and you'd get the same chunking of the material. The thing which you guys thought to make in a paper. It's a hypothesis. Well They are doing these things together. But one of the things that you one could imagine doing is um sen is, you know, having the transcription there and pl just play it back to a bunch of captive undergraduates and say when do they stop and start Um because because essentially you're looking at um cycling sequences in discourse. And if the discourse tells you what's going on that's information. I mean I g one of the questions is how the information is gonna be shared across these media. And if you can get it all out of the speech, you know, if the if all of the chunking is available when there is speech, okay, then it's carrying a lot of the burden. It's saying we are done with that and Mm. But this is but we haven't looked at data like this with this with our old-fashioned analyses um, you know. And i it's it would be really nice to know that that much information, that chunking of the task information is being carried by the language. Because J_P_'s question, the overriding question is so what's language for. You know, if people are busy interacting all the time and all of our colleagues don't even bother to control for whether people are talking to one another when they're doing these joint tasks because it seems to them to be irrelevant 'cause language is irrelevant, it would actually be nice to demonstrate that in a place where we're controlling whether you have language or not. You could get the entire chunking of the task out of a language. Mm-hmm. Mm-hmm. Mm-hmm. Because you know it's a self-contained system as opposed to a system which is which can't be interpreted without the other system. Okay. Usually the claim. Almost all studies on language claim that language is a self-contained system which will give you everything you need to know. That's the claim. Alright. I didn't say it was true. I said that's the claim. And so it's worth testing. Mm-hmm. Mm-hmm. Mm-hmm. No no. And we won't and establishing the full set will, you know, take us much of the project. So you know Fine. My only concern w uh you a as I said when I barged in was just to make sure that we didn't lose the things that we might need to pick up later. That's all. Every single frame. Well Okay. But since the l the definitions of look a thing we're looking at are d um are the ones which get rid of the lower level. Like jiggling around in the area of a particular object, right. So if we're moving the the green triangle um we've defined a region which is the dynamically the green triangle, wherever it is. We will jiggle around in there. But it doesn't matter where we are in there. That's beneath our level of analysis. S Well the in that interpretation is definable to frame rate by our by the only way we're gonna use frame rate. I c well, I mean i because I can I can say for how many frames this fixation went is officially defined. So that as long as I as long as it's not a untimed event, okay. A and an end time. So it's, you know, it's fine. I can say it's a long one. Or it's a short one. Or it's, you know, it's twenty milliseconds more than that one. That's Mm-hmm. Mm-hmm. And okay, I'm I'm asking you as an expert. This is this this isn't a leading question. This is a question question. Why would that be informative particularly? Uh-huh. Mm-hmm. Uh-huh. On it. So you think that if for example if they're looking at um one or another um apex of a triangle it would just that would make a difference. They were exploring the thing, as opposed to they were simply alright, there is it there it is. Okay, so th um so that really means that we're not throwing away absolute fixations. The screen location of absolute fixations. Alright. Uh-huh. As opposed to out of it. So you so I thought I understood you to mean exploring the figure. Because you know, your fixation is a point. And the figures are bigger than points. So you could be exploring the figure or you could be just somewhere in the region defined as the figure but not on it. Okay. Alright, fine. The So you just want to know the variability of the of the fixation. And wouldn't that differ from person to person? Okay. So it's the number of different spots within that region where the eye has fixed the number of different fix oh, ok okay. So it's a jiggle rate, okay. Right. Mm-hmm. Well, all I th understood you to ask for was the duration of each fixation inside. Yeah. You can come back. Okay. Right. So it's the number of different fixations. Well, actually do you want the average duration? Or do you want the number of different ones? Duration of each fixation in the region. Well, you've already got that. So that's a measure of jiggle in the region, roughly. Moving. Okay. What's our error of measurement on location on the screen? So to what degree do we actually know whether the two little eyes, little circles, are exactly in the same place or just somewhere in the same region? Right, okay. Right. Mm-hmm. It's yeah, okay, right. S Okay. So my question was going to be alright, the the mouse is a dynamic object, right. The eye track is a dynamic object? Or is it only the the piece that the eye tra the other person's eye track is on that's that's an object in this definition? Is a is an object. Okay. With some with plus or minus something or other, yeah, around it. Some circular area. Okay. And Um so you can be on the object but not on the other person's gaze. So there can be a triangle which has A_'s gaze in the centre and your gaze actually happens to be here. You're in the triangle region. Does this Okay. But it's directly added when when these two in this situation when these two overlap, right, like this. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Well it would come up it would say yes, right? I mean because they would be in the same place. So if you y you're gonna have dynamic regions which overlap one another by definition. If the mouse is on the construct, then right, right. Right. We treat those as as other objects. But the difference between them and the the parts of the c thing to be constructed is that they're allowed to overlap with other things so nothing breaks. Okay? If we mm-hmm. Ye Mm-hmm. Yes, as opposed to simply looking at somebody's gaze when the mouse isn't also there, right. And you want to know those things. Right, they're i they're independent dynamic objects. Okay. So a definition of we're all looking at and touching the same thing. So imagine that we've just added a piece to the construct, alright. So now that's the construct. And both mouse symbols are on it, right. And both gazes are on it, right. So at that point you should have a line-up of A_ is on the construct, A_ is on the mouse, A_ is on A_ is on A_'s mouse, A_ is on B_'s mouse, A_ is on B_'s gaze, and B_ likewise. Okay. Right. Ooh, I love the eraser. Mm-hmm. Yeah, but the mouse is also a dynamic object. That's fine. Okay. Mm-hmm. Mm. Mm-hmm. Right. And then you then you look for combinations of them. Yeah. I think we need to. Because tha some measure of alignment is looking where the other guy is directing attention. And there are two measures of where the other guy is directing attention that you get from the visual track. One is where the mouse is and the other is where the gaze is. So Mm-hmm. Right. Right. So Yeah, that actually true. Be 'cause Right, but that it's also gonna be the case that um whatever region you define as the region of the dynamic object of the red triangle may get to the point where it overlaps the region of the dynamic object of the green square. Okay, what do we do about that? Yeah, because if you have them close together, there's always some fuzz factor around them. And so if they're left lying close together, okay, or in the in fact in the model, in the not in the model, in the in the supplies set they're actually neatly packed into a little space. Mm-hmm. Mm-hmm. Squeak squeak. Yes. Because the two are so close together that their regions overlap. Mm-hmm. The greatest overlap with what? Ho ho. I think I think you're throwing information away. Yeah. But it may be a bad idea. It may be a bad idea. You've just thrown data away. Yeah. J Well The gaze will wobble. The gaze will wobble. That's natural. Mm-hmm. Or indeed they might be looking from one to another. D or deciding which one to choose or thinking so you don't want to decide it's only one and throw the other one out. Mm-hmm. Okay, well that's fine. That I was going to say that's the obvious way to do it. Well each one is yes or no. couple of dozen. Right. Six to a dozen usually. Mm-hmm. Yeah. I mean I think it's very likely that um if people are considering f mm-hmm. Well, not if the regions overlap, okay. If the regions i Right. And if you define the regions so neatly that there's that, you know, that we're losing gaze because of jiggle outside this closely defined region when it's in the middle of space, then I mean I I actually like the solution, however ugly it it looks in terms of a data format of having a track for every possible object of gaze. Mm-hmm. S a thing. Mm-hmm. Well okay. So Mm-hmm. Uh-huh. Mm-hmm. Mm-hmm. I always thought that was brilliant. Yeah, it is big, and you do have to scroll through it to see what's going on. But it gives you a very clear picture of what's going on. Yeah, well, okay. But so we'd we'd better do a test one and and listen to them scream. With parts. Right. So I mean so Mm-hmm. Okay. So what about the point where a part becomes the construct? Do you still identify it as a part? I mean I think one should lose that I take it. 'Cause Mm-hmm. Mm-hmm. Right. So you cha so every time a part is added to the construct, it becomes a new part. So the construct actually um has say it has six things that are added to some initial thing, okay. So if there's just the initial thing you've put it in the middle of the screen, that's just the initial object, right. Soon as you put a part, there's a construct. But it's construct one. Because when you add another part, construct one ceases to exist and you get construct two, right. So actually defining all of those as the construct is gonna be the trickiest thing. Right. So you're you're looking at triangle one, right, and all of a sudden it ceases to exist. 'Cause you can build sub-assemblies. And then you can start again. Also when you when you screw it up, you g start again. Because the definition Mm-hmm. Well, the problem is that all of these things are popping out of existence. So do we know the difference between um we broke it and it went away and um it's now part of the construct? Can we ta can we tag the constructs with what's in them? Yeah. Okay. So we could we could essentially well, then we could cumulati or it is a new construct 'cause it has a new list of a new cumulative list of parts, right. But we will Right. So So some fancy programming's gonna have to be done to say I was looking at triangle one and now I'm looking at triangle one in construct one. 'Cause the first construct was triangle one and square one, okay. Alright. So now we have to ask whether um So l let's imagine this, we've made a construct of two parts, and we're c i a triangle and a square. And we're considering now adding something on to the side of the square that isn't attached to the triangle. Okay. Are we looking at the square or we looking at the construct? Yeah. Uh in this definit should be looking at the construct. There isn't a way to figure out that you're looking at the square rather than the triangle. Mm-hmm. It is. Well Okay, because I mean that's that's the example m f presumably that's the best example of overlapping overlapping areas. 'Cause if if the thing the two s the two things are now abutted. Of course there's an area which is common to both of them, okay. Um so what I'm trying to run through in my head is that we can always tell the difference between um something going out of existence 'cause it's joined to construct something going out of existence 'cause we threw it away, right. Suppose we pick up an we we we screwed up the first time we put the triangle with the square and we threw it away and decided we didn't like it, and we took another one. Is it triangle one? Is it triangle one prime or something like that? It's the replacement for triangle one. Mm-hmm. Well how do I know what it replaces? N n Uh this is true. But if I have triangles one two three, right, and I throw them away, any triangles I take out are to replace one two three. Okay. Except that ex no, you don't know which one they're meant to replace. But you know they're not meant to replace any that are all that are that haven't been thrown away. Okay. Right. So the new parts are replacing those which have disappeared from the screen. And if there's no more T_R_ one track the T_R_ one track is gone now. Oh there are many. Same identity. Oh, is that true? You've decided not to make all the triangles red and squares green and Oh, okay. Right. One two and three. they Okay. So So there So shape plus colour. Okay. They're unique. All the parts are unique. So wai hang on. That makes the task a little bit easier, right. The uh all the lines. So Indeed. Which kind of leaves you with an obvious way of making the same task harder and easier, doesn't it? But but that's hard to code. I mean that's really a thing we might do. Okay. So now let's make sure that the coding would survive that. It's just it's a beautifully controlled situation. The construction task is essentially the same. It's the figuring out stuff that gets harder if everything is purple. For a just to choose a colour at random here. Mm-hmm. Mm-hmm. Mm-hmm. So if you have five black triangles, there gonna be five black triangles in the parts box? Or one? Uh-huh. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Okay. As long as it this is possible. So c that that just is a kind of thing which is so simple, so much like a single one zero variable change that it if it that doesn't already exist. 'Cause I'm tr I'm trying to kinda map this onto Baufix. Um and Baufix has has multiple, you know, all the the The the v the nuts are are red or and all the um flat things are wood colour something. The same part the same part is different colours. Oh yeah? So when you say a red nut or a green bolt or a long green bolt then there'll be lots of them lying around. And I think that's how the robot is set up to there are five five green bolts. Uh here is one, you know. Okay. Okay. So Okay, so w uh all I'm I'm doing is kind of worrying out loud about all the things that will happen that we'll miss by th a simple view of for example regions and looking at at triangle one. When is triangle one not triangle one anymore. Um does it s keep its identity. Mm. Identity, yes. Mm-hmm. That's true. So how can we define the shapes that people build? Well If assuming I was for a moment I was J_P_ and I wanted to know how they actually went about building the thing. Okay. And I wanted to um to look at the strategy. And I wanted to see if the strategy was different when we could talk about it and when we just picked up whatever. I mean suppose I can't talk to you and I'm doing this task with you and I can't talk to you. Um the thing which is gonna be hardest for me is making elaborate plans with you. I can reach for the next thing and you can go where I reach. But if I have some, you know, sub-goal, some long-term sub-goal, of doing something clever with putting these together 'cause it's hard and then putting these together 'cause it's hard, there it's gonna be almost impossible to convey that to you. And the difference in the history of construction, right, is an important thing, and I wonder how we can get that information back. Do you think? No, it why is not uh that's that's not the question I'm asking. The question I'm asking is what's the history of the construct? So you have the n the names of the things in there. And you have two constructs suppose we have two sub-constructs. And whichever one had two pieces put together first is the earlier numbered of those constructs, right. How do you tell the difference between adding a piece to this first construct and creating a second construct? Mm. Okay. It's it's it's parent child relationship. Okay. So construct two could b um construct one is um a red triangle and a green square. Construct two is um two green squares, okay. Construct three, right, is a red triangle, a green square and another red triangle. Construct four okay. Okay. And you can so how are we gonna tell when for example you think we're gonna have to do human coding on when two sub-constructs are put together. No, that's a definition wherein a construct has a c has constructs as children? Okay. Okay. Right. So we could zip we could zip through this and look at all the um all the interactions in which people build sub-construct contstructs first which we expect them to do bu as a wild hypothesis more when they have verbal communication than when they don't, okay. And we can do that by simply searching for any constructs that have constructs as children. Okay. Mm-hmm. Yeah, we haven't actually made a rule that you can't uh collect extra parts, right? No no. I mean you have to b you have to break one to get one, okay. And you can't just bring extras in case you screw up. You have to screw up first and then you can bring an extra. Mm-hmm. Mm-hmm. Mm. Mm-hmm. I but it does have replacement. Mm-hmm. Mm-hmm. There there's a so you so so you appreciate you appreciate the cost of screwing up. Mm-hmm.. Mm-hmm, that's right. Right. Okay. So they rejected something or they broke something inadvertantly. Okay. Is there a way of telling the difference between intentional and unintentional breakages? Bec yeah. No, you p they both put they both put their hands on a construct because they don't like it.. Mm-hmm. Yep. So that just has to be coded. Mm-hmm. Um not generically. Because they'll if they're gonna do it by gesture they'll build up a convention, right. What else do we need to know? Unless they develop a convention. Suppose they start off with language and then they they say hey look, you know, let's do it like this. Sure. Absolutely. I just I just wanna make sure that a wish list, yes. Yes. For sure you know that's and l or well n is not necessarily intentional. They could just screw up. Yep. Mm-hmm. Or I don't think that's very good, no, we'll never get away with that, we'll get a bad score. Let's just throw that away and start again, yeah. Okay. Which reminds we have fixed the score problem. Have we? Right. Was the percentage overlap and that made the score? Yeah? That's when I left. Oh yeah, well do you know which which which which one was the top right. Yeah. Mm-hmm. But that's all been fixed. That's that's that's back. Don't tell me it was a feature. Alright. Oh okay. Oh. So Hmm. But The right. Not prominently flagged. Okay. A communication flaw. Okay. So we can now that's now just lovely. And there won't be any crazy scores anymore. Oh. I have to use a new page. Alright. So the last I think I'm sorry, you know, I had as assorted emails from you and I was probably didn't catch everything. The problem was that there are two Camtasia records. They don't see their own gaze. They see the o only the other guy's gaze, right. Huh huh. Okay. I mean Check out Okay. So we Mm-hmm. Is there mm. Is there such a thing um as um, you know, dual track video? Because you can certainly mix things. Um what ideally one wants one doesn't really want to l to have these things. Uh um because we're using them for back-up and for coding. We don't really want to play them independently. We do almost always want to use them in exactly in parallel. Time aligned. Um what are we doing for synchronization otherwise? Okay. So that we'd have to re-align them. We couldn't dump them from their start points. Well, maybe you could. The thing that you're dumping them onto just starts running with the flash and the bleep. You turn on the two copies whenever at different times. But they're still no? Is the what what i th what are they I don't really understand the the parts that are operating here. Two videos are turning on. There is no time stamp that comes from some common source. Mm-hmm. Right. Right. Okay. Right, is there a div mm. Mm-hmm. Mm-hmm. Mm-hmm. So they'll include things like the um the eye tracker calibration, right, which is not a bad thing actually. Because sometimes you wanna go back and find out if this was just a particularly duff subject.. Right. Yeah. And hand every time we re-calibrated the damn thing slipped, yeah. That actually helps. I mean that ni Right. Yep. Mm-hmm. Right. No, they that actually saves you hours trying to make somebody's data smooth out when it won't. Because yeah. That's a technical term. That's a a technical categorization in psychological research. And that's because? Well we need sound we we need real sound recording, right. That's that's two track. That's stereo. Alright. Uh-huh. On w on what? Uh sorry. Uh On one of the Con O o it's uh On Camtasia. That's our only sound record? Mm. Mm-hmm. How good is it? Yeah. Have to really question. Mm-hmm. And then then they can do sound analysis on that. Oh okay. Right. So we there's gonna be a way to, sooner or later, to align the eye track, Camtasia one, Camtasia two with two sound channels. Right. Mm-hmm. Ah, the bleep we'll find. Mm. Right. Right. Right. They uh they also tend to have a funny funny shape, beeps. So Mm-hmm. Mm-hmm. Yeah, is there an end signal? When you decide it's all over, does it ping you back and Right. Okay. So when you've when you've pressed we're finished, it pings back at you? Okay. It's just that all kinds of crazy things happen when you're running things, and so up saving a percentage of a trial is sometimes a good i you know, being able to do that is sometimes a good idea. Okay. Can w Yeah, can we do can we do something better than that? Like having Yeah. Yeah, I mean it's it's crazy. We're in an anechoic room which we specially, yeah, built an anechoic. But, you know, sound room which we specially built for this. We're d in good circumstances. We shouldn't screw it up more than we have to. Is there anything we can do to improve this? No. No, no. Do you have a powered microphone or something. Right. That the line is stereo, the input is mono. Right? Is that what you said? Right. Is ah. That's somewhere in the sound card. Yep. Its own track, its own video. You have to align the videos anyway, right? I Well I'm not the technical person who would do it. But I know who to ask. Mm-hmm. Mm-hmm. But it's the same as Mm-hmm. Says Robin thinking about cutting and splicing, yeah. Yeah, okay. Mm-hmm. Mm-hmm. Yeah, I wanna see several of them. I wanna see I wanna see one of them um look really nice before we start running subjects. Can we do that? Yeah. Yes, that's right. I just wanna know it works the way you expect it to work. This is not a a gesture of mistrust. This is just experience that um if anything can screw up it will. I I'm trying to choose my language carefully because we're being recorded. But you'll hear more choice language at the point when we've all done a lot of work and and then we discover we can't use the session because of some thing we didn't think about. Right. Um Mm-hmm. Mm-hmm. Mm-hmm. Right. And then then we combine Yeah. It's probably worth mentioning this to the to the guys in the garage in Toronto, right. To say mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Uh-huh. Right. I think that's a if the documentation said it w could record in stereo then I think we should talk to them about how they've done this. Is you're not sure from the documentation what it says it can do? Mm-hmm. Right. Okay. Okay. Mm-hmm. Yeah. Right. Right. Okay. Right. Right. Right. Okay, so when you I mean uh given that there are these events recorded, we can use any of them as the beginning of any of them as a start point, for example, right. Um there are also motions being recorded of objects. Right. Mm-hmm. Right. Mm-hmm. Okay.. Right. Yeah. So I'm right. Uh that that that's great. I I'd before we break up for lunch I just wanna make sure that tha that I know how long it's gonna be before we're in run mode, okay. So the sound thing the sound thing is stands in our way, we need to be recording sound online. Okay. So that's a thing that has to be solved. The shape thing is solved. Mm-hmm. We think. We believe. Right. So that's that that that should be done this week. Um in fact like tomorrow for example. Um the shape all the shape problems are solved. Generating shapes, no problem. Um scoring shape overlap, no problem. Nothing is problem there. There are no visual display problems. Mm-hmm. Yeah, they had their They had their inspection last week. Um J_P_ staggered away from it and Ah. Mm-hmm. Okay. S Um I haven't I haven't had any discussions with him since. Well, he's ph he's physically here. If we want to pay him for some more time we can do that. Would that be a just better use of our time in get Yeah. Right. But m he he he is around and I believe. And if if the if i if it's a Right. But if um if everybody has a lot to do and, you know, Joe could give it a couple of hours, which i is what it might take just to fix it, then let's do it because starting run-time is now getting to be fairly urgent. Mm-hmm. Mm-hmm. Right. Right. But we can't we can't make Joe do this ec alright, so there's uh some trade-off between availability and speed from start to finish, given that this is somebody else's code. So you know. Right, so we're we're b we're building models. Right. We're we um We have the design somewhere. Mm. Hmm. Well Mm-hmm. Okay. Yeah. I think so. 'Cause um I have to check back with with J_P_ fairly soon. Okay, he wanted to kn um The eighteen the next eighteen month plan, great. Deliverables, great. Progress with the experiment, well that's what we're doing now. Okay. So what do we think, if you have a fairly hectic week, are we looking for subjects next week? Okay. Ads go up. Yeah, I do. I do. But I think ad should probably go up this week. We want people for n for next week. Is that okay? Is anybody now terrified at the thought that we're live next week? There are people coming in here one after another. Or p two two after two, more likely. Yeah. Yeah. Okay, is the is this is this generalised is this generalised anxiety, Robin? Or do you have a specific thing that okay. So aside from the usual angst which we all suffer when we go live. So we w we wanna do some piloting at the end of the week? Right. Okay. Right. S Right. She's using but she's actually relaxed that somewhat. Yeah. Mm-hmm. Mm-hmm. Ostensibly booked time, which isn't Okay. We've but we have we have a solid booking for all the mornings anyway. Right? And l except by permission of us, alright. We own it. Um so we can change that. I mean they everybody acknowledged the the other week that we that we had priority. Oh yeah. Yeah, yeah, yeah. Oh yeah, yeah. Oh yeah, yeah. Don't worry. Um there's there's a booking system, right. So it's public what's booked and what isn't. Okay. No. Alright, maybe w should get on there and book mornings as far as the eye can see. Alright. Uh-huh. Yeah, just a well we think we spoke to all the people who who used those facilities the other day. But we could be wrong. So Huh. Right. Um well the two key holders are here. Oh I see. From Caroline? Or the general office? The general office. I see. But no, they need a they need an account on this machine to actually to use it. They need account on these machines so that they could book. They couldn't actually log in. Mm-hmm. Okay. We're also working on a way of making sure that the matin that the machines return to zero state when they're when you come off them. because the it's the people have been leaving var um well, people have been leaving various bits of who knows what around on them, right, which which we think is what's critical. Okay. Um piloting piloting piloting. You have we have to build the things. We have to try the machi th things. We have to check them with Marloes and J_P_ that they like the representation, the variables, right. How about uh you have any time on Friday? I get a email back. Maybe you and I should be the pilot subjects. First pilot subjects, the ones who um know which questions we wanna ask. And then we should get another pair who are naive to the whole thing. Okay. This is generalised floating anxiety. No no. It's it's it's wisdom. But yeah. Well um because actually we can I mean uh, you know, if we make comments, they feed back in a lo into the development.
Uh five hundred Hertz though. Mm-hmm. Yeah. Uh m m i uh i yeah but the p sort of start and end times will give you back to that output. A as a and then you just measure it against, you know, the c the c yeah, the cumulative total time record or something. So Mm-hmm. B uh uh m yeah, so I'll I'll lag in some of it during the I suppose there's the the saccadic movement itself, for example. Th th th yeah, the it should still come out that the little sketch I asked this p for for last week about N_X_T_ and whether it would have that sort of tiered effect so you could see the overlap of where one person is looking compared to the other and their mouse movements and things. That that should ra uh address all that. Mm-hmm. But but because it's also in the i based on the the time course of the procedure you'll be able to basically get the scan path of the pattern of events from that as well. Yeah, yeah. So I did have it somewhere. I didn't have it r uh haven't brought it. trying to remember it. Mm-hmm. The Uh it m it m uh but uh br at that point they're all of it lumped all the individual fixations as long as it doesn't move off that triangle together as a total sort of gaze gaze time on that. Yep. Yeah. That that's a target within the target region. Uh m Uh-huh. M yes, somewhere. Mm-hmm. It it sh yeah, it I mean uh as long as we're dif r yeah, it there shouldn't be anything missing. That should be alright... Yeah. Yep. M m yeah. Yeah, yeah, yeah. Mm-hmm. Yes. So we yeah. So we if we have an an event, yeah, that that yeah, the second line then for the second before it. And then we can just take that chunk out and do something with it. That should I I Right. Yeah. Mm-hmm. Yeah. Yeah. Uh well, if it's just based on uh like a a stable fixation for so long or or something and crossing into that region. Yeah. yep. Mm-hmm. Yep. Mm-hmm. Mm-hmm, yeah. To mm-hmm. Don't know. Mm-hmm. More. Mm-hmm. Mm-hmm. Well he's really interested in Mm-hmm. So which one does Mm-hmm. Mm-hmm. That's what I have understood it to be. But well I don't know. But W uh mm-hmm mm-hmm. Yes, uh-huh, yeah. It's mm-hmm. Mm-hmm. You can we can announce now. Mm-hmm. Mm-hmm. Well Uh but y y y You mean to the tags of the data w within it. Yeah. So so things like fixation, the number of fixations a and Bi um yeah, with some some measure of the so we could work out the number of fixations that made and the average fixation duration as well. Because having st long steady fixations can be informative. Um if if they're holding their if there's less dancing around, the um the cognitive focus tends to be in one sort of part. So uh there is a difference between if they even though they're still looking at the same part, if they're looking around it uh rather than just holding their gaze steady or um and longer on it. Uh yeah. It, yeah, it's it's certainly it's it's worth mm-hmm. Uh no, I wouldn't want to throw out Put the actual location Mm-hmm. Mm-hmm. I mean presumably Uh if well yes, but that probably isn't going to be easy to get out because you'd then have to break up uh the parts somehow. Yep. Mm-hmm. Uh it it yes, it can do, but it's also looking at it within their their own behaviour to see, yeah. Uh th w I I would say just the number of fixations. I mean it could just be yeah, the number of different yeah. Yeah. Mm-hmm. Yep. Exactly. So Mm-hmm. I mean to It yeah. Uh if if we've got everything, then I suppose it's you can always yes, it's you're not throwing anything away then. Or Right. Mm-hmm. Uh but uh uh number uh well the uh duration time as well somehow. Whether it's the average one or Of the fixation. Well The the average should be alright uh to work with. The the Uh well, if you've got the number of fixations in the average time, you can then just generate it. Yeah, yeah. Mm-hmm. Uh Again, uh that that should be you should be able to derive that presumably from Right. Uh that it can vary a bit depending on the calibration. But if we're building in a sort of error margin um it it will be based on pixels around. But there'll also be a minimum one. 'Cause I'm trying to think that it's probably the small the actual mouse icon is so small, it'll need a slightly larger uh just because if what you have i is a sort of larger error or of b error margin around it. Uh no. The o the other person's gaze position as well. Mm-hmm. Yep. Trea treated the same as as the mouse say. Yep. Mm-hmm. Yes, yep. Yes. So if there uh if mm-hmm. Yes. But not looking at their individual and pr fact, they probably will avoid looking directly. Because if they're looking directly at it, it's obscuring the the part. Mm-hmm. Yeah, mm-hmm. The Mm-hmm. Mm-hmm. The m the mouse, the gaze and the part are all layered. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Uh uh if when they get moved together. Mm-hmm. It Mm-hmm. Mm-hmm. Two objects close together, your eye gaze can actually be looking at the left edge of one and the right edge of the other. And they're sor therefore looking at two objects at once. Saying they need some new pens. New pens. White-board pens. Mm-hmm. Uh unle uh but it uh unless we just define it as there's only they're they are exclusive. And which ever area m has the greatest overlap is what they're actually looking at. Well it w if if you've got your sort of eye position, and it's it's unlikely to be exactly fifty percent in one object and fifty percent in the other. Uh Well I uh uh put these things on. Yeah. Uh uh what I'm I'm thinking is if you've got say one object there and right. You know, so this is thank you. Uh part one and part two. And you've got they're sort of they're looking there. Then that overlap means they're looking at the two pieces simultaneously. But Yeah, uh just if this is there's more of it uh yeah, I yes, uh that's w uh that's that's a decision we'll uh uh uh I'm saying is if they're looking if most of what their overlap is here then we stick with that one. Uh that's th we no, that because um the gaze position will be a sort of fuzzy area. It won't just be a pixel. It'll it'll be an actual like circle and area. Yeah. So i i it it that's that's going to be w uh it's it's not just a single point. It's it's a it's a sort of lump target. And so there'll be a bit of a a sort uh some sort of overlap of two objects. Or b yeah, they could just be oscillating mm-hmm. Mm-hmm. Uh I mean if if they are oscillating though, then that's presumably going to st change the weight. So you are actually going to see them flipping between uh when you decide they're looking there mm-hmm, if they're really close, yeah. Mm-hmm. H yes. Hmm. The yeah. Mm-hmm. Mm-hmm. Uh i it's also going to be a problem uh because you're going to have one part or a construct forming in the middle of a fixation. So it's not gonna be quite as clear if you can't can't chop it off. Uh-huh. And then it yes. And it becomes construct or assembly whatever two. No. You can build sub sub-assemblies and then link the the part really, yeah. I so each Wai w I th uh Which constituent parts. Construct. You'll be looking at the construct. 'Cause that's the way it's dis defined. Looking at areas within it. Hmm. Uh w They can't replace uh the new parts only appear when the previous one's broken. Yes. They ca they can appear without them. You c Uh with same shape but not the same shape and colour. Uh same eye. Yeah. Yep. Well that's well that's that's what they're at the moment. Uh Yes. So that's the way it works at the moment. Uh I just assume that's how ni Nijmegen had set it all up. Th that's that's back to uh Tim's day. They are individual uh you have different colours for the yep. Unique, yeah. Yes. The tam tamgrams are much easier when you can see Mm-hmm. Yep. It is just a construction task rather than a puzzle. Yep. Mm-hmm. Yeah. Uh yes. It at the moment there are d uh there are two I think two small ones and two large triangles. They're actually the same physical size and shape. At the moment they're in different colours. That also means in the p new parts list they're there for, you know, two v large triangles and two small one small ones, say. Whereas in in the revised edition if they were all just black there'd only be one uh mold in the new parts. As far as I know it's hard-wired. It just appears so. Mm-hmm. The polygon one, polygon two, whatever it might be. Mm mm-hmm. Yep. Hmm. Yep, same same pieces. Mm-hmm. No, so they they can have different uh nuts there. W well nuts that are different colours. But same part in different colours, yeah. No, what I mean there's more than they're multiple pieces that look the same. Mm-hmm. Mm-hmm. Mm-hmm. Yes. Mm-hmm. Mm-hmm. And the extra parts don't appear. So you can't collect them. Yeah. There there are a couple of them that it's like greyed out. They really very faint. So you see the mold is there. But they don't come filled uh until Yes. Yes. I i it appears in the new p uh like uh suddenly the th th the new part lights up sort of thing and it's there. Yes. They have to bring it into play. No. Nope. That yeah. That that is one think unlike the say the Baufix or anything. We I assume the the Baufix. The there aren't any extra parts. And at the moment the software doesn't a allow for that. Yes. Replacement, but no superfluous so you can't have a a standard stock with extra parts that it'll never use. Mm-hmm. Mm-hmm. Yeah. Mm-hmm. Yeah. Mm-hmm. Intentional um in the sense that they break deliberately when they bring them into the new parts. But they c you know, it's like the trash can idea that they bring a You have to hope that there's language and one of them says something. You c mm-hmm. Uh there's no there's there's no way you could tell that just from the the raw movements or or something. Mm yep. Yeah. It I Although What I'm trying to remember is if the people deliberately break things by moving two pieces together. They they don't both break. They both they both Mm-hmm. So the i intentional thing is when they actually both click on the same object and it breaks. No, they could just make a mistake or screw up, yeah. Y yeah. But that's bringing in the the language thing again. Mm-hmm. Mm-hmm. Oh uh yes. At the the percentage score. Yeah. So our yes, rather than the number of penalties or or something. No, the the score w uh is fine. Uh ye that's that's the way it's worked out. But uh uh the problem was um to do with the m the p they had a relative and absolute piece and uh it's all to s to the symmetry of that. So as long as it's defined, fine. Yes. It wasn't it wasn't a bug. It it it was a Uh well. Uh yes, it was something that had to be defined and the uh Dutch lot didn't. So that was it was actually their their fault. But um it's not clear enough in the documentation. Yeah. Yeah. But i it it is quite fundamental and uh causes quite immediate problems if you don't know about it. Yep, exactly. Yeah. Or oh we're well we yeah. we should should go back to the beginning. The um Camtasia was tested last week and initially we were a bit worried because uh in the old version it seemed to interfere with it. But uh when you when Craig re-compiled it and did his new version it started wor wor it seems to work fine. Um Nothing we can tell. No uh Mm-hmm. No. Yes. There will now have to be two Camtasia records because uh e each person actually sees a different screen because of the gaze feedback. They don't see their own gaze to the other person. So there'll need to be two separate Camtasia videos generated. Hopefully we won't. You mean officially? Or uh Yeah, I hadn't thought about the I was assuming we'd just get one and one licence both of us. Suppose technically we should have two because it'll be two machines. I suspect that it will. It'll be per machine I would think. Hmm. I I'll have a uh yeah, I'll have a l a look at the actual Camtasia licence. Um the other thing we're we It 'cause it's now twice twice the size of a Uh mm. Hmm. Mm-hmm. Mm-hmm. Mm-hmm. Well the other problem is uh Camtasia has to be manually launched on both machines separately. Yeah. Uh during the task there is a flash and a bleep. Oh I mean not during the task. Uh before. For each task. Yes. Yes. Uh Camtasia y yeah. But No, they're they're on two s the two separate machines and the you you start and end them separately. The they they're not talking to each other. They're just recording. You start it recording and it starts recording what's on one screen. And you start the other recording on the other machine. And they're they're totally separate. They don't know about the other one. Mm-hmm. Yep. Yes. Uh yeah. See how how long it took them actually. That's that's true. How long did it take them to calibrate. Or yeah. How many t how many times before uh the validation worked or something. Yeah, that's true. I uh Well it at the moment the that's what I was assuming on on on doing that uh it's done first. Mm-hmm. You go back and actually yes, they were crap. Isn't that the answer? Yeah. Um yes. The other thing we were getting sorted out was the the microphones. 'Cause only one of the uh Camtasia videos will actually have the soundtrack. Uh we're just getting we've got one cable and to the two mics. I suppose we can do it is it would it be worth splitting it? Uh yeah. So the at at at the moment it's ins well, what we've finally just uh at twelve o'clock today looks like we've got it sussed um is to get one channel one microphone being the left channel and one being the right channel to merge into a stereo single file. Uh unfortunately the on one machine. The display P_C_s. Yeah, the Camtasia v video. Or m m multi-media thing. Yeah. Yes. It it's Yep. It's Camtasia that's recording it.
Yeah. Yeah. Yeah. Yeah. Yeah. So so m so I assume that well once you've that sort of information, you well you got the G_D_F_ format. You then filter it to say, right, between time X_ and time Y_ they're they're constructing that having triangle one to the construct. And then you just say in that time period, what percentage are they looking at triangle one. What percentage are they looking at the construct. Does that sound does that sound reasonable? Yep. Yeah, Right. Right. Yeah, all all the all events have a start time and an end time. So Yeah. Yeah. Yeah. Yeah.. Okay. Yeah 'cause I think that that's gonna co that's 'cause b the diagram we got there suggesting that they're only looking thing at time. So whether it's triangle or whatever else. But so what you're suggesting there is they can be looking at the triangle and maybe a mouse pointer and maybe an eye at the same time. Okay, okay. Yeah. Yep. Yeah, yeah. 'Cause you get c 'cause you Yep. Yeah. Yep. Yep, yep. Okay, yeah. So so yeah. So so it's just a matter of adding these, the two mouse pointers and the um two gazes as extra objects. Um so yeah. As long as as long as they've got an I_D_ in there we can just say right at this p between these times they're looking at that object or they're looking at the mouse, they're looking at whatever. Yep. Yeah. Pens. I confuse 'em. Yeah. that would Yeah. Yeah. Well thing is that with Yeah.. Yeah. thing thing is W_G_D_S_ formats is wor is working at the moment. It's actually got a separate track for each part. So Yeah. Well the Well the the the it depen depends on the depends on what you build in. There's different parts there's different parts whether you're doing the tamgram or whether you're doing an airpa airplane or whatever. Um But th but sa but thing is if if we if we got if we got tracks for each for each part we can we can easily analyse it down later on. Yeah. Yeah, well the Yeah. Yeah. Well the the the the doesn't really list part in the in the G_D_F_. Um so yeah. That that that's not a problem. There so that bas basically what I've got is well there there there's places where you got the I_D_ though. So you've the an I_D_ for a part, you got an I_D_ for a location on the screen. So it shows you the target, config the clock, whatever else. Um and then we can easy add I_D_s for the two mouse points and the two gazes. Um and from that and ba basically all all the all the tracks are doing is saying at this between this time there's a look at this o this object. Yeah, th this I_D_. Um so uh so it's effectively a track for each I_D_. The th yeah the but but Well wha wha what happen what happens there is that the the two the the existing parts basically cease to exist and a new part with a new I_D_ starts existing at the point where they're joined. Yep. Yeah. Right. Yeah. Oh. Yeah. Yeah, that that that that's what that's what that's the way he's working the moment. Because it uh th it's it's generating a new I_D_ when you c when you create a new part. Um Yeah. Um yeah. But there there's there's joint events. So if the if a joint event if if if it's there's joint event linked to linked to the item then you know it's become part of a construct. Um of um Yeah, the the the yeah. The con the con the constructs are already tagged with that. So that I'm act there's actually a s a separate file of the data format which says for each construct, which which two parts made it made that came together to make that construct. Yeah. But you can still ta you can still track what parts went into it. So you can still find triangle one, whatever. Yeah. Yeah. But but that's easy n uh it's easy to do because the the co each construct has the I_D_s for the pa the things that construct it. Yeah. Yeah. Well we'll d yeah well with the yeah d well thing is that we we still have the X_ wa the X_ wave traces from the ASCII formats. So if if you want if you want to be able to say I'm looking at a square instead of I'm looking at the construct, that's it is possible to get that data out. Um it's it's yeah. It's it's not doing it's not doing it in the gest analyzer at the moment. But I can easily add it to give that information if you want. Yeah, if if you take it if you t if you crea if you create a new part, it uh it gets a new I_D_. So if you get if you drag a new part out of the parts bin Uh Yeah, I mean the we e Yeah, I mean you know whi you know which way they match. So you can Yeah, the Yep. Yeah. Yeah. So yeah. Yeah, so I think that anything's loading. There's there's then two triangle, isn't there? There's two identical triangles in the tamgram. Um Well yeah yeah. When you when you specify the molds in the in the in the p the file that c that generates the parts, like a list of parts, the molds are bas just taken out of the the file that generate that specifies the parts. So so ho however many part however many mo however many parts there are in the file, that's how many parts appear in the in the parts box. If you have five black triangles gonna be five five parts in the fi five black black triangles in the parts box. But what what what but what you can do well if in the in the initial configuration you can re-use a triangle. So if you so if you only want one black triangle, you just create one black triangle in the parts box and then re-use that pa that triangle five times in the configuration. Yep. Yep. Well when the when the f when the final construct the the way the the way the the constructs are defined in the file, you've got like your p your parent construct and the parts that it's constructed out of. Um and yeah, as I say, the Yeah, yeah. So you got parent child relationship and i the child can be a part or it can be another construct. Yeah, yeah. 'Cause uh there's a joint action. The result the t what you need for a join action you need to yeah, a construct and a part, two parts, whate whatever you're taking together. And then the result of that is a construct. You've got time when that happens. You can see that construct appeared at that time. Um and you can still track the children of that construct. Yep. Yeah. Yeah, well by by the by the time by the time they're doing that they must have already broken something. So you'll so you'll know what they're replacing. So Uh they do. If if if what if one if one person's trying to get a part to a part that isn't that that so the part that's just sitting on the screen that's not being touched. They both break. Yeah. Yeah. Yeah. Yeah. Uh well yeah. 'Cause yeah. C 'cause it was through the rotational part. Yeah. Yep. Yeah. Yeah, I mean I'm I've th I've yeah. It's it's in the documentation. But it's like two thirds of the way down a quite a long page. So Yep. But um Yeah. Yeah. No. No the the the there's there's very small graphical bugs. But it's it's not it's not big thing. It's like couple of white pixels uh in the course of a in the course of a the course of a task. So it's there's distracting. No. Yeah, but then the que the que the question is there a licence for Camtasia where you just need one editing c thing and two recording things. 'Cause yeah, the r the recording yeah, the recording thi pro program is just quite a it's quite a small program. They just gen it just generates the video and so I don't know don't know if it's worth having a look at the licensing for that just to see. Yeah. Yep.. Mm. Yeah. No, you got you Yeah, you got exactly the same access to sound quality as you got from anythi anything else on the Windows. So you can you decide whatever quality you want. So got a bit of Uh yeah. Because the the synchronization, the audio and vi the audio visual synchronisation comes at a specific point in the eye track file. No, it's yeah th it's i i sta stamped into it. So it's just after just after it prints the line starting experiment uh uh yeah. I'll I'm pretty sure it's just yeah. Yep. Uh well the well the MATLAB script uh bas just sa uh just says uh yeah, gi gi gives you the times of the audio beep in the s in the signal. So I just Yeah. Yeah. Oh yeah. Yeah. Pretty confident. But you you've you've seen them. It it was it was Yeah. Yeah, I don't see him with ten he kilohertz. Yeah. Yeah, the the the there's a signal at the start and the end of every trial. So Yep. Yep. Yes, it's a stereo jack. It's a stereo jack. But it's a mono but it's a mono uh microphone. Mono mono recording on microphones. Line in. Uh sp going going through a line in. So it's a so it's a lit so it's quieter because it's not as though it's not powered. But it's still, think, yeah. We we still pick up a signal. Yeah. Yeah. Well the the when they are they are the the they have batteries in them. But i there's not doesn't. But doesn't seem to be powered because the because the line in input's so low. The so the sound the sound the s Well the yeah, the the yeah, the the the the jack that is that it's plugged into is stereo. But when you plug in when it's recording from microphone, it records in mono. And yeah. A new sound card is fixable. Yeah. Als think we we have a well I suppo I suppose it well one th one thing you do is uh re is record the sou speech signal separately for each for each person. But then you'd have to pull the two pull the two sounds together. So if uh so if i i if each machine recorded its own with a recorded the recorded like the the p th the uh the user on that one. Yeah. Yeah. Oh they yeah. And then the so the yeah. So th Yeah, it's easy. Easy. If if nothing if nothing else you h uh if nothing else you can do it in a MATLAB script. You just write read read file one, read file two. And then you just yeah. Yeah, but then the the the MATLAB the MATLAB script will tell you how l it will tell you how far apart they are. 'Cause you can do you can just measure the drift between the two of them. 'Cause you got a l you got a list of where the where the beeps are. And then you say right, this stream is like, I don't know, nought point two five seconds behind the other one. Yeah. Mm-hmm. Yeah. So then They've both got the same sound card. So yeah. Yeah. Yeah. 'Cause 'cause 'cause Yes. Yeah. So the the external version's got plugs that size and that that is stereo microphones. But yeah. The other one's plug's like the smaller size and that isn't stereo. So the colour the colour can do it. It's just that the connectors on the the connectors at the back of the computer can't. Yeah. But as as far far as we can and find the the web site of the, yeah, the the Creative Labs web site and the review of it. Um and they weren't entirely clear. But they both suggested that it is there is a mono that there there is a mono microphone. Yeah. Yeah, I'm Yeah. So and basically out the output the G_D_F_ is just gonna be a whole series of events, you know, with start and end time. And the event's gonna have whatever all whatever the object uh is that's being looked at and things. Um and then, yeah, and then it's a matter of we're gonna put whatever filters you need after that. So you can say right, I just want so you can say from the time where they picked up triangle one to the part time it became part of the assembly, find out how it's how m percentages of time they were looking at it or whatever. Yep. So yeah. So you got mo yeah. There there's move events. There's look events. there's fixations and blinks now at the moment as well. Um and, yeah, and oh yeah and see uh looks at looks at objects and looks whatever else. So um s Yep. Yep. Yeah. Yeah I will.. Yep. Yep. Okay. Fair enough. Okay. Well yeah. Yeah. Yeah. Yep. Yeah. So we'll we'll get we'll get the get the audio stuff tested, you know, single microphones. Um yeah. Get that tested next couple days and I assu I assume that's not something you need me for t in terms of that.. You don't you don't need me involved in the pilot, do you. So well unle unless unless it's a bug report, yeah. I'll w yeah w I'll I'll I'll b I'll be around. But yeah, I'll make sure. But ho hopefully it will run smoothly and yeah, I can relax. So
